Papers with LLM frameworks
MEBench: Benchmarking Large Language Models for Cross-Document Multi-Entity Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) and Retrieval-augmented Generation (RAG) systems show promise, but their performance on cross-document MEQA remains underexplored due to the lack of tailored benchmarks. |
| Approach: | They propose a scalable multi-document, multi-entity benchmark to evaluate LLMs' capacity to retrieve, consolidate, and reason over scattered and dense information. |
| Outcome: | The proposed benchmarks show that even advanced models achieve only 59% accuracy on MEBench. |
CypherBench: Towards Precise Retrieval over Full-scale Modern Knowledge Graphs in the LLM Era (2025.acl-long)
Copied to clipboard
| Challenge: | Graphs are used for storing open-domain knowledge and domain-specific enterprise data. |
| Approach: | They propose to use property graph views on top of the underlying RDF graph to efficiently query LLMs. |
| Outcome: | The proposed graph views can be efficiently queried by LLMs using Cypher . the proposed graphs have a large schema, overlapping and ambiguous relation types and lack of normalization. |